Storing and identifying metadata through extended properties in a historization system

ABSTRACT

A historian system enables the creation, storage, and retrieval of extended metadata properties. A tag metadata database module of the historian system stores tag metadata received over a network connection in a tag metadata database and retrieves tag metadata in response to requests received over the network and from within the historian system. An extended property database module creates extended properties associated with a tag metadata instance in response to requests, stores the created extended properties, and retrieves the stored extended properties in response to requests. The extended property search index module indexes extended properties as they are created, searches the indexed extended properties in response to requests, and provides the indexes of extended properties to enable location of the extended properties in the extended property database.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/460,756, filed Jul. 2, 2019, which is a continuation of U.S.application Ser. No. 14/833,906, filed Aug. 24, 2015, which is acontinuation-in-part of U.S. application Ser. No. 14/789,654, filed Jul.1, 2015, which is a continuation-in-part of U.S. application Ser. No.14/704,661, filed May 5, 2015, which is a continuation-in-part of U.S.application Ser. No. 14/704,666, filed May 5, 2015, which claims thebenefit of and priority to U.S. Provisional Application No. 61/988,731,filed May 5, 2014, and U.S. Provisional Application No. 62/092,051,filed Dec. 15, 2014. The entire contents of the above identifiedapplications are expressly incorporated herein by reference, includingthe contents and teachings of any references contained therein.

BACKGROUND

Aspects of the present invention generally relate of the fields ofnetworked computerized industrial control, automation systems andnetworked computerized systems utilized to monitor, log, and displayrelevant manufacturing/production events and associated data, andsupervisory level control and manufacturing information systems. Suchsystems generally execute above a regulatory control layer in a processcontrol system to provide guidance to lower level control elements suchas, by way of example, programmable logic controllers or distributedcontrol systems (DCSs). Such systems are also employed to acquire andmanage historical information relating to processes and their associatedoutputs. More particularly, aspects of the present invention relate tosystems and methods for storing and preserving gathered data andensuring that the stored data is accessible when necessary.“Historization” is a vital task in the industry as it enables analysisof past data to improve processes.

Typical industrial processes are extremely complex and receivesubstantially greater volumes of information than any human couldpossibly digest in its raw form. By way of example, it is not unheard ofto have thousands of sensors and control elements (e.g., valveactuators) monitoring/controlling aspects of a multi-stage processwithin an industrial plant. These sensors are of varied type and reporton varied characteristics of the process. Their outputs are similarlyvaried in the meaning of their measurements, in the amount of data sentfor each measurement, and in the frequency of their measurements. Asregards the latter, for accuracy and to enable quick response, some ofthese sensors/control elements take one or more measurements everysecond. Multiplying a single sensor/control element by thousands ofsensors/control elements (a typical industrial control environment)results in an overwhelming volume of data flowing into the manufacturinginformation and process control system. Sophisticated data managementtechniques have been developed to store and maintain the large volumesof data generated by such system. These issues are compounded in asystem that stores data from multiple tenants at once in such a way thateach tenant's data is secure from access by others and enables users tocreate and define custom data properties to be associated with theirdata. It is a difficult but vital task to ensure that the process isrunning efficiently.

SUMMARY

Aspects of the present invention permit creation and retrieval ofextended properties associated with tag metadata instances of processdata. Moreover, aspects of the invention provide secure storage of theextended properties and efficient indexed searching of the extendedproperties.

In one form, a historian system enables the creation, storage, andretrieval of extended metadata properties. The system has a clientaccess module, a tag metadata database module, an extended propertydatabase module, and an extended property search index module. Theclient access module receives and responds to messages over a networkconnection. The tag metadata database module stores tag metadatareceived over a network connection in the tag metadata database andretrieves tag metadata in response to requests received over the networkand from within the historian system. The extended property databasemodule creates extended properties associated with a tag metadatainstance in response to requests, stores the created extendedproperties, and retrieves the stored extended properties in response torequests. The extended property search index module indexes extendedproperties as they are created, searches the indexed extended propertiesin response to requests, and provides the indexes of extended propertiesto enable location of the extended properties in the extended propertydatabase.

In another form, a method for creating extended metadata properties in ahistorian system is provided.

In yet another form, a method for retrieving extended metadataproperties in a historian system is provided.

Other features will be in part apparent and in part pointed outhereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram detailing architecture of a historian systemaccording to an embodiment of the invention.

FIG. 2 is an exemplary diagram of a historization workflow performed bythe system of FIG. 1.

FIG. 3 is an exemplary diagram of the structure of the system of FIG. 1.

FIG. 4 is an exemplary diagram of cloud historian abstraction layersgenerally according to an embodiment of the invention.

FIG. 5 is an exemplary diagram describing a metadata server in relationto the rest of the historian system of FIG. 1.

FIG. 6 is an exemplary diagram describing tag metadata caching accordingto an embodiment of the invention.

FIG. 7 is an exemplary diagram describing the dependencies betweenelements of the Historian system.

FIG. 8 is an exemplary flowchart describing the process of storing datain the Historian system.

FIG. 9 is an exemplary flowchart describing the process of retrievingdata from the Historian system.

FIG. 10 is an exemplary diagram of the structure of a historian systemcomprising elements for implementing extended metadata properties.

FIG. 11 is an exemplary diagram an extended metadata property database.

Corresponding reference characters indicate corresponding partsthroughout the drawings.

DETAILED DESCRIPTION

Referring to FIG. 1, a distributed historian system, generally indicatedat 100, enables users to log into the system to easily viewrelationships between various data, even if the data is stored indifferent data sources. The historian system 100 can store and use datafrom various locations and facilities and use cloud storage technologyto ensure that all the facilities are connected to all the necessarydata. The system 100 forms connections with configurators 102, datacollectors 104, and user devices 106 on which the historian data can beaccessed. The configurators 102 are modules that may be used by systemadministrators to configure the functionality of the historian system100. The data collectors 104 are modules that connect to and monitorhardware in the process control system to which the historian system 100is connected. The data collectors 104 and configurators 102 may be atdifferent locations throughout the process control system. The userdevices 106 comprise devices that are geographically distributed,enabling historian data from the system 100 to be accessed from variouslocations across a country or throughout the world.

In an embodiment, historian system 100 stores a variety of types ofinformation in storage accounts 108. This information includesconfiguration data 110, raw time-series binary data 112, tag metadata114, and diagnostic log data 116. The storage accounts 108 may beorganized to use table storage or other configuration, such as pageblobs.

In an embodiment, historian system 100 is accessed via web roleinstances. As shown, configurators 102 access configurator web roleinstances 124. And data collectors 104 access client access point webrole instances 118. Online web role instances 120 are accessed by theuser devices 106. The configurators 102 share configuration data andregistration information with the configurator web role instances 124.The configuration data and registration information is stored in thestorage accounts 108 as configuration data 110. The data collectors 104share tag metadata and raw time-series data with the client access pointweb role instances 118. The raw time-series data is shared with storageworker role instances 126 and then stored as raw time-series binary data112 in the storage accounts 108. The tag metadata is shared withmetadata server worker role instances 128 and stored as tag metadata 114in the storage accounts 108. The storage worker role instances 126 andmetadata server worker role instances 128 send raw time-series data andtag metadata to retrieval worker role instances 130. The raw time-seriesdata and tag metadata is converted into time-series data and sent to theonline web role instances 120 via data retrieval web role instances 122.Users using the user devices 106 receive the time-series data from theonline web role instances 120.

FIG. 2 describes a workflow 200 for historizing data according to thedescribed system. The Historian Client Access Layer (HCAL) 202 is aclient side module used by the client to communicate with historiansystem 100. The HCAL 202 can be used by one or more different clientsfor transmitting data to historian system 100. The data to be sent 208comes into the HCAL 202 and is stored in an active buffer 210. Theactive buffer 210 has a limited size. When the active buffer is full214, the active buffer is “flushed” 216, meaning it is cleared of thedata and the data is sent to historian 100. There is also a flush timer212 which will periodically cause the data to be sent from the activebuffer 210, even if the active buffer 210 is not yet full.

When historizing 226, the data may be sent to a historian that is onpremises 204 or a historian that stores data in the cloud 206 (step228). The HCAL 202 treats each type of historian in the same way.However, the types of historians may store the data in different ways.In an embodiment, the on-premises historian 204 historizes the data bystoring the data as files in history blocks 230. The cloud historian 206historizes the data by storing the data in page blobs 232, which enableoptimized random read and write operations.

In the event that the connection between HCAL 202 and the historian 204or 206 is not working properly, the flushed data from the active buffer210 is sent to a store forward module 220 on the client (step 218). Thedata is stored 222 in the store forward module 220 in the form ofsnapshots written to store forward blocks 224 until the connection tothe historian is functional again and the data can be properlytransmitted. The store forward module 220 may also dispose of data aftera certain period of time or when it is full. In those cases, it willsend an error to the system to indicate that data is not being retained.

FIG. 3 is a diagram displaying the historization system structure in aslightly different way from FIG. 2. An HCAL 306 is hosted on anapplication server computer 302 and connected to a historian computer304 and a store forward process 308. The HCAL 306 connects to thehistorian through a server side module known as the Historian ClientAccess Point (HCAP) 312. The HCAP 312 has a variety of functions,including sending data received from HCAL 306 to be stored in historyblocks 320. The HCAP 312 also serves to report statistics to aconfiguration service process 314 and retrieve historian data from aretrieval service process 318.

The HCAL 306 connects to the store forward process 308 through a storageengine used to control the store forward process. The Storage Engineenables the HCAL 306 to store and retrieve snapshots and metadata 310 ofthe data being collected and sent to the historian. In an embodiment,the store forward process 308 on the application server computer 302 isa child Storage Engine process 308 related to a main Storage Engineprocess 316 running on the historian computer 304.

In addition, HCAL 306 provides functions to connect to the historiancomputer 304 either synchronously or asynchronously. On successful callof the connection function, a connection handle is returned to client.The connection handle can then be used for other subsequent functioncalls related to this connection. The HCAL 306 allows its client toconnect to multiple historians. In an embodiment, an “OpenConnection”function is called for each historian. Each call returns differentconnection handle associated with the connection. The HCAL 306 isresponsible for establishing and maintaining the connection to thehistorian computer 304. While connected, HCAL 306 pings the historiancomputer 304 periodically to keep the connection alive. If theconnection is broken, HCAL 306 will also try to restore the connectionperiodically.

In an embodiment, HCAL 306 connects to the historian computer 304synchronously. The HCAL 306 returns a valid connection handle for asynchronous connection only when the historian computer 304 isaccessible and other requirements such as authentication are met.

In an embodiment, HCAL 306 connects to the historian computer 304asynchronously. Asynchronous connection requests are configured toreturn a valid connection handle even when the historian 304 is notaccessible. Tags and data can be sent immediately after the connectionhandle is obtained. When disconnected from the historian computer 304,they will be stored in the HCAL's local cache while HCAL 306 tries toestablish the connection.

In an embodiment, multiple clients connect to the same historiancomputer 304 through one instance ofHCAL 306. An application engine hasa historian primitive sending data to the historian computer 304 whilean object script can use the historian software development kit (SDK) tocommunicate with the same historian 304. Both are accessing the sameHCAL 306 instance in the application engine process. These clientconnections are linked to the same server object. HCAL Parameters commonto the destination historian, such as those for store forward, areshared among these connections. To avoid conflicts, certain rules haveto be followed.

In the order of connections made, the first connection is treated as theprimary connection and connections formed after the first are secondaryconnections. Parameters set by the primary connection will be in effectuntil all connections are closed. User credentials of secondaryconnections have to match with those of the primary connection or theconnection will fail. Store Forward parameters can only be set in theprimary connection. Parameters set by secondary connections will beignored and errors returned. Communication parameters such ascompression can only be set by the primary connection. Buffer memorysize can only be set by the primary connection.

The HCAL 306 provides an option called store/forward to allow data besent to local storage when it is unable to send to the historian. Thedata will be saved to a designated local folder and later forwarded tothe historian.

The client 302 enables store/forward right after a connection handle isobtained from the HCAL 306. The store/forward setting is enabled bycalling a HCAL 306 function with store/forward parameters such as thelocal folder name.

The Storage Engine 308 handles store/forward according to an embodimentof the invention. Once store/forward is enabled, a Storage Engineprocess 316 will be launched for a target historian 304. The HCAL 306keeps Storage Engine 308 alive by pinging it periodically. When data isadded to local cache memory it is also added to Storage Engine 308. Astreamed data buffer will be sent to Storage Engine 308 only when theHCAL 306 detects that it cannot send to the historian 304.

If store/forward is not enabled, streamed data values cannot be acceptedby the HCAL 306 unless the tag associated with the data value hasalready been added to the historian 304. All values will be accumulatedin the buffer and sent to the historian 304. If connection to thehistorian 304 is lost, values will be accepted until all buffers arefull. Errors will be returned when further values are sent to the HCAL306.

The HCAL 306 can be used by OLEDB or SDK applications for dataretrieval. The client issues a retrieval request by calling the HCAL 306with specific information about the query, such as the names of tags forwhich to retrieve data, start and end time, retrieval mode, andresolution. The HCAL 306 passes the request on to the historian 304,which starts the process of retrieving the results. The clientrepeatedly calls the HCAL 306 to obtain the next row in the results setuntil informed that no more data is available. Internally, the HCAL 306receives compressed buffers containing multiple row sets from thehistorian 304, which it decompresses, unpacks and feeds back to the userone row at a time. Advantageously, network round trips are kept to aminimum. The HCAL 306 supports all modes of retrieval exposed by thehistorian.

FIG. 4 shows a diagram 400 of the components in each layer of ahistorian retrieval system. The hosting components in service layer 402include a configurator 408, a retrieval component 410, and a clientaccess point 412. There are simple processes that are responsible forinjecting the facades into the model and have minimal logic beyondconfiguration of the libraries and expose communication endpoints toexternal networks. The hosting components could be the same or differentimplementation for cloud and on premises. In FIG. 4, there are threeintegration points for cloud and on premise implementation. A repository414 is responsible for communicating with data storage such as runtimedatabase or configuration table storage components. A client proxy 416is responsible for communicating with run-time nodes. An HSAL 426, whichis present in runtime layer 404, is responsible for reading and writingto a storage medium 406 as described above. The service layer 402further includes a model module 428.

In addition to the HSAL 426, the runtime layer 404 includes a componentfor event storage 418, a storage component 420, a metadata server 422,and a retrieval component 424.

In an embodiment, for tenants and data sources, the repositories 414serve as interfaces that read and write data using either page blobtable storage or an SQL Server database. For tags, process values andevents, the repositories 414 act as thin wrappers around the clientproxy 416. In operation, the client proxy 416 uses the correctcommunication channel and messages to send data to the runtime engine404. The historian storage abstraction layer 426 is an interface thatmimics an I/O interface for reading and writing byte arrays. Theimplementation is configurable to either write to disk or page blobstorage as described above.

In an embodiment, the historian system stores metadata in the form oftag objects. Every historian tag object is a metadata instance, whichcontains tag properties such as tag name, tag type, value range, andstorage type. Moreover, the tag object is uniquely defined by a tag ID,which is a 16-byte globally unique identifier (GUID). The storedmetadata includes values that determine how the associated data valuesare stored. This includes metadata that indicates whether the associateddata value is a floating point value, an integer value, or the like. Themetadata includes, for example, an engineering unit range that indicatesa range in which the associated data value must reside for theparticular engineering units being used. In an embodiment, the historiansystem makes use of the engineering unit range to scale the raw datavalue when storing it on the data server. For instance, data values maybe scaled to values between 0.0 and 1.0 based on the engineering unitrange included in the metadata. Because the metadata contains theengineering unit range, the scaled value stored by the historian can beconverted back to the raw data value with the added engineering unitsfor presentation to user. For example, if the data value is of a datatype known to only return values between −10 and 30, a data value of 30is scaled to 1.0 and a data value of −10 is scaled to 0.0. A data valueof 10 is scaled to 0.5. As a result, the scaled data values as stored onthe data server cannot be interpreted correctly without knowing therelated metadata in order to convert from scaled value to true valuewith the appropriate units.

The concept of tags is different from the concept of tag metadatainstances. A tag is identified by a tag name, while a metadata instanceis identified by tag ID. So for the same tag the system can have severalmetadata instances sharing the same name, but having different tag IDs.For example, the same tag could be reconfigured several times along theway. It could be created first as 16-bit unsigned integer, collect some16-bit data, then reconfigured to be 32-bit unsigned integer, collectsome 32-bit data, then reconfigured to 32-bit float. In this example, itcomprises a single tag but has three different tag metadata instancesidentified by tag ID. A tag metadata instance can also be called a tagversion. Tracking tag metadata is essential for data processing and,advantageously, the historian tracks what is stored in the raw binarydata chunks. The historian stores tag versions in two places: A tagtable (and its dependent tables) of a runtime database stores the mostrecent tag metadata called the current version, and the history blocks,where, for instance, tag metadata for classic tags is stored in filestags.dat, and for the other tags in files taginfo.dat.

When a tag is reconfigured over time, the runtime database maintains thecurrent version. All previous versions can be found in the historyblocks where previous versions are stored.

A Metadata Server (MDS) according to aspects of the invention is amodule responsible for tag metadata storage and retrieval. FIG. 5 showsa diagram 500 describing the relationships of the MDS 508 to othercomponents of the historian. An HCAL 502 is connected to the historianby HCAP 504 as described above. A storage engine 506 receives data fromthe HCAP 504. A retrieval module 510 accesses data from the storageengine 506 and metadata from the MDS 508 to retrieve it in response toqueries. The storage engine 506 stores data in history blocks 514 anduploads pre-existing tag metadata to the MDS 508 on startup. All tagversions are stored in the Runtime database 516 for modem tags. Forseamless backward compatibility, the storage engine 506 discovers filesin history blocks 514 and uploads all found tag versions into MDS 508.Internally, MDS 508 maintains two containers in memory indexed by tag IDand tag name. The two containers in this embodiment comprise the runtimecache and the history cache. The runtime cache contains all tag metadatapresent in the tag table of the runtime database and its dependenttables for modem tags. The MDS 508 subscribes to runtime database 516change notifications via a configuration service 512 so if tags areadded or modified in the runtime database 516, MDS 508 immediatelyupdates its runtime cache to mirror the tag table.

A diagram 600 of FIG. 6 illustrates the relationship between an MDS 602cache and a runtime database 604. A runtime cache 606 interacts with ahistory cache 608 within the MDS 602 by deleting and resurrecting tagsas necessary. A tag table 610, which keys on tag names, and a taghistory table 612, which keys on tag IDs, interact with each otherwithin the runtime database 604 by similarly deleting and resurrectingtags as necessary. The MDS 602 synchronizes the caches 606 and 608 withthe tables 610 and 612 within the runtime database 604. The runtimecache 606 is kept in sync with the tag table 610. The history cache 608is kept in sync with the tag history table 612. When tags are deleted orresurrected between the tables 610 and 612 in the runtime database 604,the caches 606 and 608 are synchronized to reflect this change.Synchronization also works the other direction, with changes in thecaches 606 and 608 occurring in the tables 610 and 612.

If a tag is requested to be deleted, it is moved from the runtime cache606 to the history cache 608. A reverse process is called tagresurrection, causing the MDS 602 to search the history cache 608 tofind a tag metadata instance with all the same properties and a tag IDwhich can be reused again. The runtime database 604 implements a similarlogic. Instead of generating a brand new tag ID it tries to reuse theexisting one from the tag history table 612 and move the correspondingtag record from the tag history table 612 to the tag table 610.Advantageously, the tag resurrection logic prevents generating anunlimited number of tag metadata instances in scenarios when the tagproperties are periodically changed.

FIG. 7 illustrates the dependencies and relationships of various modulesin the historian system in the form of a diagram 700. In an embodiment,the described modules in diagram 700 comprise processor-executableinstructions for fulfilling the purpose of the modules. At the userlevel, the historian system comprises an Online Web Role instance 702for end users accessing historian data from different locations,On-premise Data Collectors 704 for monitoring and gathering data fromthe historian system from on the premises, and On-premise CollectorConfigurators 706 for configuration administration of the historiansystem.

The Web Role instance 702 connects to a Data Retrieval Web Role module708 to retrieve tag metadata and time-series data from the historian. Inan embodiment, the Data Retrieval Web Role module 708 comprises an ODatalayer. The Data Retrieval Web Role module 708 connects to both aMetadata Server Worker Module 714 to retrieve tag metadata 720 and aRetrieval Worker module 716 to retrieve data by tag name.

The On-premise Data Collector 704 connects to a Client Access Point(CAP) module 710 in order to create tags and send time-series data tothe historian for storage. The CAP module 710 also connects to theMetadata Server Worker module 714 to create and retrieve tag metadata720 and the Retrieval Worker module 716 to retrieve data by tag name,and further connects to a Storage Worker module 718 to store rawtime-series binary data 724.

The On-premise Collector Configurator 706 connects to a Configurator WebRole module 712 for registering on premise data collectors with thehistorian and other configuration tasks. The Configurator Web Rolemodule 712 connects to the Storage Worker module 718 for reading andwriting configuration data 726 to the database.

The Metadata Server Worker module 714 creates and retrieves tag metadata720 in a memory storage device of the historian database. The MetadataServer Worker module 714 retrieves metadata and provides it to the DataRetrieval Web Role module 708, the CAP module 710, and the RetrievalWorker module 716. The CAP module 710 also provides new tag metadata tothe Metadata Server Worker module 714 to write into the tag metadata 720in the database. Additionally, the Metadata Server Worker module 714writes diagnostics log data 722 to the database as necessary.

The Retrieval Worker module 716 of FIG. 7 retrieves tag metadata fromthe Metadata Server Worker module 714 and raw time-series binary datafrom the Storage Worker module 718. In an embodiment, the RetrievalWorker module 716 decodes the raw time-series binary data using the tagmetadata in order to provide requested data to the Data Retrieval WebRole module 708 and the CAP module 710. Additionally, the RetrievalWorker module 716 stores diagnostics log data 722 on the database asnecessary.

The Storage Worker module 718 reads and writes raw time-series binarydata 724 in a memory storage device of the database and providesrequested raw time-series binary data 724 to the Retrieval Worker module716. Raw time-series binary data is received from the CAP module 710 andstored in the database. The Storage Worker module 718 receivesconfiguration data 726 from the Configurator Web Role module 712 andwrites it to the database, while also retrieving configuration data 726from the database and providing it to the Configurator Web Role module712. Additionally, the Storage Worker module 718 stores diagnostics logdata 722 on the database as necessary.

In an embodiment, the historian system maintains data for multipletenants such as different companies and the like. The data fromdifferent tenants should be securely isolated so as to prevent access ofone tenant's data by another tenant. The historian system providessecure data isolation by making use of the described tag IDs and tenantspecific namespaces. Each tenant namespace is made up of uniquelyidentified tag names within the namespace itself, and that tag names areassociated with tag IDs as described above. In an embodiment, the tagIDs are unique identifiers such as universally unique identifiers (UUID)or globally unique identifiers (GUID).

The tag IDs are used to identify tag names and also tag types, raw dataformats, storage encoding rules, retrieval rules, and other metadata. Acombination of tag metadata properties uniquely identified by a tag IDis called a tag metadata instance, as described above.

In an embodiment, the historian system uses the divide between raw dataand metadata to enforce access security of multiple tenants to the rawdata. Storage of the data in the historian system occurs through aseries of steps as described by the flowchart in FIG. 8. In anembodiment, the steps are carried out by one or more software modulescomprising processor-executable instructions being executed on hardwarecomprising a processor. At 802, a tenant begins the storage operation byencoding the data value of a tag metadata instance into a raw binaryrepresentation of the data value. The raw binary representation iscombined with a timestamp and with a unique tag ID corresponding to thetag metadata instance as shown at 804. Proceeding to 806, thecombination of data is then stored in an efficient historian database inencoded form on one or more memory storage devices. In an embodiment, asingle historian database is used to store encoded data values frommultiple tenants and the metadata corresponding to the encoded datavalues is stored separately. In this way, even if a tenant gains accessto raw data that belongs to another tenant, the raw data is encoded andcannot be properly interpreted without knowledge of the metadatainstance that corresponds to the tag ID of the encoded data value.

Retrieval of data from the historian system is executed as described inthe flowchart in FIG. 9. In an embodiment, the steps are carried out byone or more software modules comprising processor-executableinstructions being executed on hardware comprising a processor. If atenant wants to retrieve all the data for a tag name in a time range,first the tenant gathers at 902 all the tag IDs associated to thedesired tag name within the tenant's namespace. A tag name may beassociated to more than one tag IDs if there are multiple versions ofthe metadata instance or the like. In an embodiment, the tag IDs arestored by a metadata server on one or more memory storage devices of thehistorian database. At 904, the tenant requests the raw binary datarepresentations for each of the gathered tag IDs within the desired timerange from the one or more memory storage devices of the historiandatabase. Upon receiving the raw binary representations, the tenantdecodes the raw data by applying the tag metadata instancescorresponding to the tag IDs to the raw binary representations in orderto interpret the raw binary representations as shown at 906. Thedecoding of the raw binary data may occur at the tenant's location orwithin the historian system if desired.

Tag metadata instances for a particular tenant are stored in a separatedatabase which is, for example, accessible by the particular tenant.This database may be located at the tenant's location or within thehistorian infrastructure. In this way, the tenant's metadata is secure.Because the metadata is necessary to properly interpret the encoded rawdata, the encoded raw data is secure while being stored in a single,efficient historian database along with encoded raw data from othertenants. Encoding of the data can include scaling of the data valuesaccording to metadata of the values as described above, or other similarencoding schemes based on the associated metadata. Because the raw dataof multiple tenants is stored together, a malicious party who gainsaccess to the raw data database will not necessarily know which tag IDsbelong to which tenant. This makes it very difficult for the maliciousparty to determine what kind of data they are accessing and whichtenant's metadata will decode the data.

In an embodiment, the data security is further enforced by a protectedaccount scheme. The protected account scheme comprises separate storageaccount keys for each tenant. Each tenant has at least one storageaccount key for accessing metadata instances in the tenant's metadatastorage account and at least one storage account key for accessing thedata values in the tenant's data storage account. The accounts cannot beaccessed without the associated storage account key. In this way,obtaining a single storage account key for the metadata instances for atenant yields no real information without the storage account keycorresponding to the associated data values. Likewise, obtaining astorage account key for data values of a tenant yields no realinformation without the storage account key corresponding to theassociated metadata instances. Storage account key data for tenants isalso maintained in a protected form requiring the use of a tenantcertificate for access.

According to further aspects of the invention, tag metadata instancesare extended with additional properties. An end user or other softwareapplication can add extended tag properties for a variety of purposes.The extended properties can be used to store additional detail about atag, make it easier to search for a certain tag, or to group tagslogically based on the extended tag property. For example, an extendedtag property identifies the manufacturer of an associated piece ofequipment or a geographic location. Extended tag properties can be usedin data queries to retrieve associated tags and data values. In anembodiment, the historian system provides interfaces enabling users tocreate their own extended tag metadata properties. Extended tag metadataproperties created by users of a tenant are configured to have uniqueproperty names within that tenant's historian data. For example, thehistorian system includes pre-defined extended properties in addition tothe extended properties that a user may define.

FIG. 10 illustrates an embodiment of a historian system 1000 withextended tag metadata properties as a block diagram. The clientapplication 1002 connects to the historian system through the HCAL 1006.The HCAL 1006 communicates with the client access point web role module710 to create and access tag metadata, including extended properties.The client access point web role module 710 connects with the metadataserver worker module 714 to store and retrieve tag metadata 720 asdescribed above. The client access point web role module 710 handlesconnections with multiple client applications. If the historian serveris disconnected from HCAL 1006, then the tag extended properties arestored locally and sent to the historian server once the connection isreestablished.

HCAL 1006 exposes methods to store and retrieve the extended propertiesfor a software development kit (SDK) 1008 and an I/O Data AcquisitionSystem (IDAS) 1010. In an embodiment, users create their own clientapplications to use the exposed methods to store and retrieve their ownextended properties. The interface uses, for example, simple objectaccess protocol (SOAP).

In an embodiment, the historian system comprises multiple types ofclients that access the extended tag metadata properties. For example,an info client 1004 is a web client that can create an extended tagmetadata property and then search existing tags using the extended tagmetadata properties as search terms. The info client usesrepresentational state transfer (REST) API's in one embodiment to createextended tag metadata properties and retrieve tags. The info client 1004connects to the historian service layer through the Retrieval WorkerRole module 716.

An extended property repository layer module 1012 enables the clientaccess point web role module 710 and the retrieval worker role module716 to add and retrieve extended properties from the extended propertydatabase 1014. In an embodiment, the extended property database 1014 isseparate from the database storing the tag metadata 720. Because theextended property data is dynamic and may easily be expanded by users,it is advantageous to maintain a separate database 1014 to store theextended property data.

In an embodiment, the extended property database 1014 contains separatecollections of data, such as in a DocumentDB database. The extendedproperty database 1014 is monitored by the supervisor role module 1016to ensure that each database collection does not exceed capacity. Thesupervisor role module 1016 enforces a rule that states when a definedpercentage of storage in a collection is consumed, one or more sets oftenant extended property data is migrated from the existing collectionto a new collection. During this migration process, any update for thattenant metadata is unavailable for a period of time. Consider an examplewhere a database collection contains the extended property data forthree tenants. If this collection reaches the threshold, the supervisorrole module migrates all the tags for one of the tenants to a newcollection in this embodiment. The supervisor role module 1016 monitorseach collection continuously and performs the migration operationautomatically based on the rule defined for migrating data. The extendedtag metadata properties are stored in a database 1014 of the historiansystem. The extended tag metadata properties are indexed to enablesearching and the index data is stored in a repository index 1018. Whena new extended property is added, the index 1018 is updated to includethe new extended property.

When an extended property is added to tag metadata, a tag extendedproperty object is queued in a tag metadata queue 1020 to be assigned toa tag worker module 1022. The tag worker module 1022 sends the tagextended property object to the search service 1024 to be indexed. Thesearch service 1024 processes the object and sends the result to theindex repository 1018. The search service 1024 updates the search index1018 with new extended properties and marks the properties as searchableand retrievable. In an embodiment, the search service 1024 only indexesstring type extended properties.

In an embodiment, if multiple clients create or update the same extendedproperty in a similar timeframe, the last update is maintained in thedatabase.

In another embodiment, when searching the tag metadata, the query ismade against all the searchable fields in the index, which includes thetag metadata as well as the searchable extended properties. Searchresults include the requested property names and the values of theproperties.

FIG. 11 is a diagram illustrating the relationships of collectionswithin the extended property database 1014. A database 1014 for thehistorian system is created under a master database account 1102. Themaster database account 1102 is secured by a master key, which isencrypted and only used by web and worker role modules 710 and 716. Thisdatabase 1014 holds one or more document collections 1104 and thecollections 1104 hold the documents 1106 containing the tag metadata.Document collections 1104 are associated at the tenant group level. Aminimum of one collection exists for a tenant group and more collectionscan be added as described above.

In an embodiment, a tenant user account 1108 for each tenant is createdand given read-only access to resources that they own. During retrieval,a tenant's user access token is provided to the system. In response, thesystem only retrieves documents that are owned by the tenant. Thepermissions for access are set at document collection level. In analternative embodiment, the permissions are set at the single documentlevel.

In some cases, tag metadata along with extended properties for multipletenants is stored in the same collection to reduce the overall resourcesnecessary for the application and minimize costs to tenants foroperating the database. Tenant metadata is identified by the tenant IDproperty in the stored documents and queries are filtered to retrievetenant-specific data.

In an embodiment, extended properties and related tag metadata arestored in the extended property database in a document format such asthe example below. One tag document per tag is created. The document IDis a hash of a Fully Qualified Name (FQN), which is a unique name forthe tag, and the tenant id.

Tag Document:

{ “id”: “hashoffqnandtenantid”, “fqn”:“DS1.ReactTemp”,“tenantid”:“t23456789”, “tagname”:“ReactTemp”, “source”:“DS1 ”,“description”:“”, “engunit”:“None”, “engunitmax”: 100, “engunitmin”:0,“interpolationtype”:“stairstep”, “messageoff”:“”, “messageon”:“”,“tagtype”: “Analog”, “property1”: “value1”, “property2”: 277 }

A tag-specific document in the first collection of the tenant group isconfigured for storing all the properties and attributes of a tag. In anembodiment, when creating a tenant, a set of pre-defined or recommendedextended properties are created. For example, all of the existing systemtag properties are created with “read-only” set to true. An extendedproperties document is created for each tenant storing a list ofextended properties, including the name of the property and the type ofthe property. Extended properties can be a variety of data types,including strings, integers, doubles, date time values, ID values suchas GUIDs, Boolean values, geographic properties, geometric properties,and the like. Property names are unique for a tenant. In an embodiment,the extended properties document also stores whether each extendedproperty is read-only. Below is an exemplary extended propertiesdocument.

ExtendedProperties Document:

{ “id”: “tenantid_properties”, “tenantid”: “t23456789” “properties”[  { “name”: property1,  “type”: “string”,  “readonly”: false  }  {  “name”:property2,  “type”: “int”,  “readonly”: false  }  ] }

In an embodiment, all the metadata properties along with the extendedproperties are stored in this manner in order to use one database forall property retrieval. Alternatively, a query option may be included toselect to query the metadata server for tag metadata. If deletion of anextended property is requested, the extended property is set to NULL inthe database so that it can no longer be searched, rather than requiringthe database to be re-indexed after removal of the extended property.

The Abstract and Summary are provided to help the reader quicklyascertain the nature of the technical disclosure. They are submittedwith the understanding that they will not be used to interpret or limitthe scope or meaning of the claims. The Summary is provided to introducea selection of concepts in simplified form that are further described inthe Detailed Description. The Summary is not intended to identify keyfeatures or essential features of the claimed subject matter, nor is itintended to be used as an aid in determining the claimed subject matter.

For purposes of illustration, programs and other executable programcomponents, such as the operating system, are illustrated herein asdiscrete blocks. It is recognized, however, that such programs andcomponents reside at various times in different storage components of acomputing device, and are executed by a data processor(s) of the device.

Although described in connection with an exemplary computing systemenvironment, embodiments of the aspects of the invention are operationalwith numerous other general purpose or special purpose computing systemenvironments or configurations. The computing system environment is notintended to suggest any limitation as to the scope of use orfunctionality of any aspect of the invention. Moreover, the computingsystem environment should not be interpreted as having any dependency orrequirement relating to any one or combination of components illustratedin the exemplary operating environment. Examples of well-known computingsystems, environments, and/or configurations that may be suitable foruse with aspects of the invention include, but are not limited to,personal computers, server computers, hand-held or laptop devices,multiprocessor systems, microprocessor-based systems, set top boxes,programmable consumer electronics, mobile telephones, network PCs,minicomputers, mainframe computers, distributed computing environmentsthat include any of the above systems or devices, and the like.

Embodiments of the aspects of the invention may be described in thegeneral context of data and/or processor-executable instructions, suchas program modules, stored one or more tangible, non-transitory storagemedia and executed by one or more processors or other devices.Generally, program modules include, but are not limited to, routines,programs, objects, components, and data structures that performparticular tasks or implement particular abstract data types. Aspects ofthe invention may also be practiced in distributed computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network. In a distributed computingenvironment, program modules may be located in both local and remotestorage media including memory storage devices.

In operation, processors, computers and/or servers may execute theprocessor-executable instructions (e.g., software, firmware, and/orhardware) such as those illustrated herein to implement aspects of theinvention.

Embodiments of the aspects of the invention may be implemented withprocessor-executable instructions. The processor-executable instructionsmay be organized into one or more processor-executable components ormodules on a tangible processor readable storage medium. Aspects of theinvention may be implemented with any number and organization of suchcomponents or modules. For example, aspects of the invention are notlimited to the specific processor-executable instructions or thespecific components or modules illustrated in the figures and describedherein. Other embodiments of the aspects of the invention may includedifferent processor-executable instructions or components having more orless functionality than illustrated and described herein.

The order of execution or performance of the operations in embodimentsof the aspects of the invention illustrated and described herein is notessential, unless otherwise specified. That is, the operations may beperformed in any order, unless otherwise specified, and embodiments ofthe aspects of the invention may include additional or fewer operationsthan those disclosed herein. For example, it is contemplated thatexecuting or performing a particular operation before, contemporaneouslywith, or after another operation is within the scope of aspects of theinvention.

When introducing elements of aspects of the invention or the embodimentsthereof, the articles “a,” “an,” “the,” and “said” are intended to meanthat there are one or more of the elements. The terms “comprising,”“including,” and “having” are intended to be inclusive and mean thatthere may be additional elements other than the listed elements.

In view of the above, it will be seen that several advantages of theaspects of the invention are achieved and other advantageous resultsattained.

Not all of the depicted components illustrated or described may berequired. In addition, some implementations and embodiments may includeadditional components. Variations in the arrangement and type of thecomponents may be made without departing from the spirit or scope of theclaims as set forth herein. Additional, different or fewer componentsmay be provided and components may be combined. Alternatively or inaddition, a component may be implemented by several components.

The above description illustrates the aspects of the invention by way ofexample and not by way of limitation. This description enables oneskilled in the art to make and use the aspects of the invention, anddescribes several embodiments, adaptations, variations, alternatives anduses of the aspects of the invention, including what is presentlybelieved to be the best mode of carrying out the aspects of theinvention. Additionally, it is to be understood that the aspects of theinvention is not limited in its application to the details ofconstruction and the arrangement of components set forth in thefollowing description or illustrated in the drawings. The aspects of theinvention are capable of other embodiments and of being practiced orcarried out in various ways. Also, it will be understood that thephraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limiting.

Having described aspects of the invention in detail, it will be apparentthat modifications and variations are possible without departing fromthe scope of aspects of the invention as defined in the appended claims.It is contemplated that various changes could be made in the aboveconstructions, products, and process without departing from the scope ofaspects of the invention. In the preceding specification, variouspreferred embodiments have been described with reference to theaccompanying drawings. It will, however, be evident that variousmodifications and changes may be made thereto, and additionalembodiments may be implemented, without departing from the broader scopeof the aspects of the invention as set forth in the claims that follow.The specification and drawings are accordingly to be regarded in anillustrative rather than restrictive sense.

1. A historian system comprising: one or more computers comprising oneor more processors and one or more non-transitory computer readablemedia, the non-transitory computer readable media having instructionsstored thereon that when executed cause the one or more computers to:store, by the one or more processors, tag metadata received via anetwork connection in a tag metadata database; and retrieve, by the oneor more processors, tag metadata in response to requests received viathe network connection or from within the historian system; create, bythe one or more processors, extended properties for tag metadata inresponse to requests received via the network connection; store, by theone or more processors, the created extended properties in an extendedproperties database; and retrieve, by the one or more processors, thestored extended properties from the extended properties database inresponse to requests received via the network connection or from withinthe historian system; index, by the one or more processors, the extendedproperties as they are created; search, by the one or more processors,the indexed extended properties in response to requests; and provide, bythe one or more processors, indexes of the extended properties.
 2. Thehistorian system of claim 1, further comprising a client access moduleconnected to a plurality of client devices via the network connection.3. The historian system of claim 2, wherein a first client device of theplurality of client devices is associated with a first tenant and asecond client device of the plurality of client devices is associatedwith a second tenant.
 4. The historian system of claim 1, wherein theextended properties stored in the extended properties database include atenant ID, wherein the tenant ID is used to filter the extendedproperties to retrieved extended properties associated with a tenant. 5.The historian system of claim 1, wherein storing created extendedproperties in an extended properties database further comprises:storing, by the one or more processors, the created extended propertiesas extended property documents in a first collection of extendedproperty documents; monitoring, by the one or more processors, aremaining capacity of the first collection of extended propertydocuments; and migrating, by the one or more processors, a plurality ofextended property documents from the first collection of extendedproperty documents to a second collection of extended property documentsupon detecting that storing created extended property documents in thefirst collection would cause the first collection to exceed itsremaining capacity.
 6. The historian system of claim 5, wherein thefirst collection of extended property documents comprises extendedproperty documents from more than one tenant.
 7. The historian system ofclaim 1, further comprising assigning, by the one or more processors, aproperty of one of searchable and not searchable to the created extendedproperties, and indexing, by the one or more processors, the extendedproperties only if the extended properties have been assigned thesearchable property.
 8. A method for creating extended metadataproperties in a historian system comprising the step of: providing oneor more processors and one or more non-transitory computer readablemedia with instructions stored thereon that implement the steps of:receiving, by one or more processors, a first request to create anextended property for a tag metadata instance; locating, by the one ormore processors, a record of the tag metadata instance; updating, by theone or more processors, the record of the tag metadata instance toinclude the extended property; creating, by the one or more processors,a document of the extended property in an extended property database ofthe historian system; and indexing, by the one or more processors, theextended property document in the search index of the historian system.9. The method of claim 8, further comprising implementing the step ofreceiving, by the one or more processors, the first request to create anextended property from a first client device coupled to the historiansystem by a network.
 10. The method of claim 9, further comprisingimplementing the step of receiving, by the one or more processors, asecond request to create an extended property from a second clientdevice, and wherein the first and second client devices are associatedwith separate tenants.
 11. The method of claim 8, further comprisingimplementing the step of receiving, by the one or more processors, thefirst request to create an extended property through a representationalstate transfer (REST) interface.
 12. The method of claim 8, furthercomprising implementing the step of receiving, by the one or moreprocessors, the first request to create an extended property through asimple object access protocol (SOAP).
 13. The method of claim 8, furthercomprising implementing the step of storing, by the one or moreprocessors, tag metadata instance record separately from the extendedproperty document.
 14. The method of claim 8, further comprisingimplementing the steps of: storing, by the one or more processors, thecreated extended property document in a first collection of extendedproperty documents; monitoring, by the one or more processors, aremaining capacity of the first collection of extended propertydocuments; and migrating, by the one or more processors, a plurality ofextended property documents from the first collection to a secondcollection of extended property documents upon detecting that storingthe created extended property document in the first collection wouldcause the first collection to exceed its remaining capacity.
 15. Amethod for retrieving extended metadata properties in a historian systemcomprising the step of: providing one or more processors and one or morenon-transitory computer readable media with instructions stored thereonthat implement the steps of: receiving, by the one or more processors, afirst request for retrieval of data comprising an extended metadataproperty; searching, by the one or more processors, a search index forthe extended metadata property; locating, by the one or more processors,a document of the extended metadata property in an extended propertydatabase based on a result of searching the search index; retrieving, bythe one or more processors, data from the extended metadata propertydocument; and providing, by the one or more processors, the retrieveddata in response to the received first request.
 16. The method of claim15, further comprising implementing the step of receiving, by the one ormore processors, the first request for retrieval from a first clientdevice coupled to the historian system by a network.
 17. The method ofclaim 16, further comprising implementing the step of receiving, by theone or more processors, a second request to retrieve data comprising anextended metadata property from a second client device, and wherein thefirst and second client devices are associated with separate tenants.18. The method of claim 15, further comprising implementing the step ofdetermining, by the one or more processors, a tenant of the firstrequest based on a tenant access token from the first request and usingthe tenant access token during the searching to locate documentsassociated with the tenant, wherein the searching requires the tenantaccess token to locate documents associated with the tenant.
 19. Themethod of claim 15, further comprising implementing the step ofreceiving, by the one or more processors, the first request forretrieval of data through a representational state transfer (REST)interface.
 20. The method of claim 15, further comprising implementingthe step of receiving, by the one or more processors, the first requestfor retrieval of data through a simple object access protocol (SOAP).